As computers have become increasingly powerful and sophisticated, it has been recognized that a primary limitation on the usefulness of computers lies in the ability or willingness of consumers to use them. It is of little value that a computer has the capability of rapidly accessing vast libraries of information if those for whom the information is intended find the process to be too difficult or tedious. Significant advances have been made in recent years in making computers more "user friendly". Predominant among these advances has been the introduction of graphical user interfaces ("GUI") which can make using computers to access information both easier and more interesting.
A somewhat related development in the field has resulted in what are sometimes referred to as "interactive" interfaces. While there is no universally accepted definition of an interactive interface, it is generally understood to be one in which the user initiates computer action by reacting to screen displays (as by "pointing and clicking" on a part of the screen display using a mouse to control the position of a cursor), whereupon the computer changes the screen display in response to the user's action and the user may respond to the revised display, and so on. This results in an "interactive" flow of communication back and forth between the user and the computer.
Interactive interfaces are used, for example, in video games, wherein the user may "click on" objects of interest in a video frame to acquire information about the object or to take some action in respect to that object. For the most part, view screens for these applications are of a "semianimated" nature wherein one or two objects are caused to move in relation to the background and/or to each other such that the computer can readily keep track of the relative locations of relevant objects and of the location of a cursor controlled by the mouse or other user input means. A somewhat similar user interface is sometimes applied to informational type applications wherein the user can click on objects in a display to get information about that object or to proceed to another video display relevant to that object. The video displays used in this type of interface are usually drawings or digitized still photographs. Examples of applications wherein such interactive user interfaces are used for such purposes can be found, for example, in a number of commercially available Compact Disk Interactive ("CDI") presentations.
In the above described user interfaces, the screen displays are generally specifically created to enable the action which the software engineers who designed the program had in mind. For example, each item in a sales display could be an active object such that clicking on that item produces information about it. In such an application, the display would generally be created after it has been decided which items should be active objects and which, therefore, should be included in the display. Such interfaces are, in many aspects, not really significantly different from the more common graphical user interfaces, except that an item defined by a designated portion of a visual scene is used as an object instead of a more conventional icon. For all practical purposes, the visual depiction of the item is an icon, and the software communicating the user's actions to the computer and responding to such actions is essentially like that of any other graphical user interface.
It should be noted that the interactive user interfaces discussed above, while being quite useful for their intended purposes, have several substantial limitations when applied to multimedia presentations. Most significant are the restrictions placed on the visual displays by the practical limitations of their method of creation. While multimedia is an emerging field the boundaries of which, in many respects, have not yet been fully defined, it is generally agreed that a multimedia presentation will employ the power of computers to provide visually and aurally perceptible stimuli to the intended audience. An objective, then, is to create video and audio that is as interesting and attractive as possible. While designers of multimedia presentations are continually striving, often with notable success, to improve the available technology and methods of production, their efforts are encumbered by an inability to sufficiently divorce the creation of the audio/video displays from the programming process.
Particularly where an interactive multimedia presentation is intended for an audience that is not particularly motivated to take part in the interaction, as might be the case in a sales presentation or the like, it is very important that the video and audio stimuli within the presentation act to capture the attention and interest of the user. However, in today's market place, where consumers are constantly assaulted with a barrage of such stimuli, it is becoming increasingly difficult to make one's presentation stand out from the crowd. As anyone who has seen the best of the motion picture special effects can attest, there are some truly spectacular audio/video stimuli available using today's special effects technology. Clearly, it would be desirable to bring such variety and dimension to interactive multimedia presentations. However, there is a definite discernible gap between the quality of presentation found in today's interactive multimedia presentations as compared to that of today's motion pictures. Even where there is an insignificant amount of interactivity required in an application, users of multimedia presentations will recognize that they are frequently disjointed and stilted as compared to a good motion picture or television show. Furthermore the content of the video, and often even more noticeably the content of the audio, while quite good as compared to the early attempts at multimedia, are still far below that found in today's high quality motion pictures. As the desired degree of interactivity is increased, even the best presently available technology will result in a product which tends to be increasingly lacking in flow and interest holding capacity.
Clearly, it would be desirable to be able to bring to bear in the creation of interactive multimedia all of the powerful tools and techniques to which modern motion pictures owe much of their attraction. However, the distinctions discussed above give evidence to the fact that this has not been possible in the prior art. Prior to the present invention, one charged with the task of creating an audio/video presentation has been hindered by the need to rigidly conform the audio/video aspects of the user interface to a predetermined series of program events. Certainly, a talented film maker encumbered by only a very few very general restrictions as to content, could create an audio/video work that would far surpass anything available today in the interactive multimedia field. However, prior to the present invention, it has been considered to be, if not impossible, at least terribly impractical to attempt to work "backward" beginning with a conventionally produced video work to create an interactive computer user interface. Among the numerous problems which have made this approach seemingly untenable is the fact that a conventional video work has no inherent reference points in time to which a computer might reference the availability of a particular action alternative.
Although a video work may have a Society of Motion Picture and Television Engineers ("SMPTE") time code associated therewith to individually identify each frame of the video this, in and of itself, is of very little use to the interactive multimedia practitioner since it would be completely impractical to try to individually treat each and every frame in the video as a separate user interface. Furthermore, the SMPTE time code provides no reference to the aspects of the video that are important to the interactive multimedia practitioner - such as the points at which there is a significant perceptible change in the visual information presented to the user. Prior to the present invention, any attempts to limit the number of video frames which are treated as user interface displays, as compared to the number of frames in a comparable conventional video presentation, have only resulted either in unnaturally interrupting the flow of the video or causing any of a number of other perceptual or cognitive lapses, most of which are sufficiently onerous as to render the method unusable.
A tangentially related concept that should be considered as prior art in relation to the present inventive method is that of a "key frame" which concept is used in the field of animation. A key frame is simply a frame of the animation that marks a transition between scenes in the animated work. The key frames are first identified in a "story board" as the animation is planned, and the animation is later completed to fill in the gaps between the key frames.
As previously mentioned, clearly it would be advantageous to allow the producers of interactive multimedia works the same degrees of freedom that are afforded the makers of conventional audio/video works. Moreover, it anticipated that it would also be useful, in many applications and instances, to convert a preexisting audio/video work, which was made with absolutely no consideration that it might later be used for the purpose, for use as an interactive computer user interface.
To the inventors' knowledge, no prior art method or means has provided a way to efficiently convert a conventional audio/video work into a practical computer user interface. All prior art methods for creating interactive computer user interfaces using digitized video and audio have either entailed creating the video and audio specifically for, and to the specifications of, the program for which they are intended, or else they have employed still pictures and/or disjointed film clips to illustrate individual aspects of an interactive multimedia presentation.